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Abstract 


The  present  article  represents  the  authors'  contribution  to  the 
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Commission  6.  It  discusses  various  topics  in  the  theory  of  discrete  information 
channels,  including  the  general  binary  channel,  channels  with  fading,  cascaded 
channels,  and  channels  with  memory. 
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Many  interesting  aspects  of  information  theory  can  be  illustrated  by 
studying  discrete  channels  with  small  input  and  output  alphabets.  In  fact, 
much  important  work  has  been  done  on  the  simplest  of  all  channels,  the 
(memoryless)  binary  symmetric  channel  (BSC).  We  refer  to  Elias'  study  of 
the  way  in  which  rate  and  probability  of  error  in  the  BSC  depend  on  the  length 
of  the  blocks  in  which  the  coding  is  done  [l] .  In  the  present  article,  we 
shall  be  concerned  exclusively  with  asymptotic  properties  of  channels,  i.e. 
properties  which  are  based  on  the  assumption  that  coding  is  done  in  arbitrarily 
long  blocks.  In  this  study,  we  shall  sometimes  consider  ternary  and  higher 
order  channels  as  well  as  binary  channels.  In  fact,  the  binary  channel  is 
too  simple  to  be  representative  of  the  general  discrete  channel,  as  can  be 
seen  from  the  fact  that  neither  input  letter  can  be  suppressed  without 
destroying  its  information  rate*. 

2.  Information  sources  and  information  rate 

The  first  concept  of  interest  in  information  theory  is  that  of  an 
information  source,  i.e.  a  device  which  generates  a  random  sequence  of  letters 
from  some  alphabet }  the  random  sequence  then  serves  as  the  input  to  a  channel 
(see  Section  3)»  Let  the  input  alphabet  consist  of  the  m  letters  ...,  x^. 

We  shall  confine  our  attention  to  independent  sources,  for  which  the  probability 
P^,  1  <  i  <  m,  that  the  input  letter  x^  is  emitted  at  time  t  is  statistically 
independent  of  which  letters  were  emitted  at  times  prior  to  t  and  of  which 

*Even  in  the  most  asymmetric  binary  channel  one  does  not  have  to  use  an  input 
symbol  more  often  than  63  per  cent  (or  less  often  than  37  per  cent)  of  the  time 
to  achieve  capacity  (see  Section  £). 
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letters  will  be  emitted  at  -times  subsequent  to  t.  Thus,  the  probability  of 
a  sequence  in  which  appears  IL  times,  1  <  i  <  m,  is  just 

N_  N 

h?2) 2  " 

According  to  Shannon  [2],  it  is  particularly  meaningful  and  fruitful  in 
studying  information  sources  to  introduce  the  concept  of  the  entropy  or 
information  associated  with  a  source)  for  a  source  S  of  the  type  under 
consideration,  this  quantity  is  defined  by 

m 

(1)  H(S)  -  -  ^  log  Pi  . 

The  base  to  which  the  logarithm  is  taken  is  conventionally  chosen  to  be  2, 
in  which  case  (1)  is  said  to  give  the  number  of  bits  of  information  per  source 
letter  (see  remarks  at  the  end.  of  this  section);  whenever  we  write  log  we  shall 
mean  log^.  Henceforth,  it  will  be  assumed  that  the  source  emits  one  letter 
per  second  (and  that  the  channel  accepts  one  letter  per  second).  With  this 
convention,  (1)  gives  either  the  entropy  of  the  source  in  bits/symbol  or  the 
information  rate  of  the  source  in  bits/second.  This  convention  allows  us  to  use 
the  terms  entropy  (or  information)  and  rate  interchangeably;  the  adjustment 
needed  in  case  the  source  emits  one  letter  every  T  seconds  is  obvious. 

We  shall  not  linger  on  the  derivation  of  (1)  from  a  set  of  properties 
which  it  3eems  reasonable  to  expect  information  to  have.  Such  derivations  are 
given  in  detail  by  Shannon  [2],  Khinchin  [3]  and  Faddeyev  [U] .  An  /Important 
property  of  H(s)  is  that  it  vanishes  if  the  source  can  emit  only  one  of  the 
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letters  x^,  x^,  and  takes  its  maximum  value  (log  m)  when  the  source  emits 

all  of  the  letters  x-^,  . ..,  xm  with  equal  probability.  (This  is  in  accord 
■with  intuitive  ideas  of  information.)  Another  important  property  of  H(s) 
is  the  fact  that  although  S  can  emit  m^  possible  sequences  of  length  N,  one  of  a 
much  smaller  set  of  2^^  sequences  is  very  likely  to  occur  if  N  is  very 
large.  (An  exceptional  case  occurs  if  all  m  source  letters  are  equally  likely; 
then  H(s)  =  log  m  and  2*^^  =  m^. )  This  so-called  asymptotic  equipartiti.cn 
property  is  fundamental  in  information  theory  and.  can  be  demonstrated  for 
much  more  general  sources  than  those  studied  here  (in  fact  for  any  stationary 
ergodic  source;  see  McMillan  [5],  Khinchin  [3]).  Finally,  we  remark  that  the 
first  of  Shannon's  two  coding  theorems  (the  so-called  noiseless  coding  theorem) 
consists  in  showing  that  precisely  H(S)  binary  digits  (bits)  per  symbol  are 
needed  to  noiselessly  encode  the  output  of  S  into  binary  digits;  in  general 
it  takes  code  blocks  of  infinite  length  to  effect  this  encoding.  The  noiseless 
coding  theorem  gives  theoretical  justification  for  measuring  H(s)  in  bits/symbol 
(or  bits/second). 

3.  Gha nnels  and  mutual  information  rate 

A  discrete  tun  memoryless  (information)  channel  i3  a  probabilistic 
device  which  accepts  any  of  m  possible  input  (or  "  transmitted”)  letters 
xl’  *•*>  \  and  emi't3  sny  °f  n  possible  output  (or  "  received"  )  letters 
y_,  ...,  yR,  in  accordance  with  the  following  rules: 

1)  For  every  input  letter  x^  and  output  letter  y^  there  is  a 
definite  number  p^,  0  <  p^  <  1,  which  represents  the  probability  that  if  x^ 
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2)  Fvery  input1  letter  gives  rise  to  at  least  one  output  letter, 


i 


3)  The  response  of  the  device  to  any  input  letter  x^  is 
statistically  independent  of  its  response  to  any  past  or  future  letter.  In 
Section  8  we  shall  abolish  this  requirement  and  consider  a  special  kind  of 
discrete  channel  with  memory. 

A  convenient  representation  of  such  a  discrete  channel  is  as  an  m*n 
channel  matrix  C,  i.e.  as  the  mxh  rectangular  array  of  numbers 


(?)  C  =* 


P11  p12  pln 

P21  p22  *•*  P2n 

•  «  • 

Pml  Pm2  “*  Pmn 


(We  shall  use  C  to  denote  either  a  channel  or  the  associated  matrix.)  By 
l)  and  2)  each  p^  lies  between  0  and  1,  and  the  sum  of  every  row  of  the  matrix 
(2)  is  unity}  these  properties  are  summarized  by  calling  C  a  stochastic  matrix. 

The  study  of  stochastic  matrices  has  received  a  great  deal  of  attention  in  the 
mathematical  literature,  especially  in  connection  with  the  theory  of  Markov 
chains  (see  Feller  [6]).  We  note  that  the  case  of  a  noiseless  channel  corresponds 
to  the  case  where  each  row  contains  one  1  and(m  -  l)0's,  while  each  column 
contains  one  1  and  (n  -  l)0*s. 

Suppose  now  that  the  input  letter  x^,  1  <  i  <  m,  is  used  with  probability 
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and  define  p(i, j),  the  joint  probability  that  x^  is  emitted  and  y^  is 
received,  by  the  relation 

p(i,j)  3  Pi3 

Then  the  probability  that  y .,  1  <  j  <  n,  is  received,  regardless  of  which  x. 

J  —  —  1 

is  transmitted,  is  given  by 

rj  *  £r ‘ 

It  is  convenient  to  represent  both  the  numbers  P. ,  1  <  i  <  m,  and  the  numbers 

,  1  “  “  ->  -> 

fy  1  <  j  <  n,  as  the  components  of  corresponding  column  vectors  P  and  P1,  i.e. 


We  now  give  a  fundamental  definition  introduced  by  Shannon  [2] :  The 
average  mutual  information  rate  (or  simply  the  rate)  of  the  channel  C  when 

it  is  driven  by  a  random  sequence  of  input  letters  chosen  independently' 

— > 

with  probabilities  given  by  the  vector  P  (equivalently,  ‘when  it  is  driven  by 

It  can  be  shown  that  for  a  memoryless  channel  any  dependence  between  input 
letter's  diminishes  the  rate  of  the  channel  (see  Feinstein  [7]). 
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an  independent  source  characterized  by  P  )  is 


-0  m  n 

r(p  )  -  n  r 


p(i»  j)  log 

n  Pi 


bits/second.  (Recall  that  by  our  convention  one  input  letter  per  second  is 
emitted  by  the  source  and  accepted  by  the  channel.) 

To  justify  (3)>  we  specialize  to  the  case  where  the  input  and  output 
alphabets  are  the  same  (so  that  in  particular  m  *  n)  and  use  Shannon’s 
correction  channel  argument  [2],  which  asserts  that 


(U)  correction  rate  +  mutual  information  rate  *  source  rate  • 

The  source  rate  is  of  course  just 


-  g  »i  log 


The  correction  rate  is  derived  as  follows:  Whenever  the  letter  y^  is  received, 
it  may  be  correct  or  incorrect.  Since  the  probability  that  y^  originated  from 
x±  is  Pj(i)  -  p(i,j)/P^,  1  <  i, j  <  m,  whenever  y^  is  received  we  must  supply 
an  amount  of  entropy 


Pj(i)  log  pj(i) 


to  correct  it  or  to  leave  it  stand  uncorrected  with  the  assurance  that  it  is 


More  precisely,  the  average  rate  of  correction  entropy. 
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correct .  Since  the  letter  is  received  with  probability 
rate  at  which  correction  entropy  must  be  supplied  is 


the  average 


m  m 

'  I*  Pi  ti  piM  los  p3(i 


Since 


m 


- 1  > 


we  can  also  write  the  source  rate  as. 


m  m 


m  m 


Pi  *ij  loS  Pi  "  ‘  g  ^  tfi,})  108  Pi 


Finally,  using  (U),  we  obtain 

mutual  information  rate  s  R(P  )  * 


m  m  mm  / .  .  \ 

“  ^  Z”  P(1,J>  log  Pi  *  g  P<±.J>  log 


m  m 


m  m 


p(i,3)  loe 

P  P 

i  3 


which  agrees  with  (3)  for  the  case  ra  *  n. 

.  In  what  follows  we  shall  find  it  convenient  to  introduce  the  symbol 
(angular  brackets)  to  denote  averaging  with  respect  to  the  joint 
probability  distribution  p(i, j),  i,e.,  if  f(i, j)  is  a  function  of  the  two 
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integral  arguments  i  and  where  1  <  i  <  m,  1  <  j  <  n,  then 

*  ZZ  ZZ  *(U5)  • 

i=l  I 

i 

i 

With  this  notation  (3)  becomes 


r(p5  *-<iogp(i)>  ♦  <iog  P,(i)>  =»<iog 

3  p,  Pi 


As  an  example  of  (3),  consider  the  general  binary  channel 


(S) 


C(a,p) 


a  1  -  a 

P  1  “P 


,  0  <  a,  p  <  1  , 


fed  by  a  source  producing  O' a  and  l's  independently,  with  probabilities  PQ  and 
P^  =  1  -  PQ,  respectively.  (Eq.  (£)  means  that  transmitted  0 's  are  received  as 
0's  with  probability  a,  while  transmitted  l’s  are  received  as  O's  with  probability 
p.)  Then  it  is  easily  verified  that  the  rate  associated  with  (5)  and  the  input 
vector 

-> 

P  - 

is 

(6)  R(a,pj 7 )  «aPolog  (a/p’ )  +  (1-a  )Polog(  (l-c)/P*)  +  pP^gCp/P^)  +  (l^logCU-p  )/*[>, 

where  P^  *  aPQ  +  {3(1  -  PQ)  is  the  probability  of  a  received  zero  and  P^  ■  1  -  P^ 
is  the  probability  of  a  received  1. 
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U»  Channel  capacity 

Following  Shannon  [2],  we  define  the  capacity  c(C)  of  the  (discrete 
raemoryless)  channel  C  as  the  largest  value  of  the  rate  which  is  achieved  when 
the  input  probabilities  are  varied  over  all  possible  values,  i.e. 

Max  -> 

(7)  c(C)  =  ->  R(P)  . 

P 

As  an  example  consider  the  BSC  where  the  channel  matrix 


is  obtained  by  setting  £  =  1  -  a  in  (5).  In  this  case  it  is  clear  from 
syranetry  (and  it  can  be  verified  by  direct  calculation)  that  capacity  is 
achieved  for  the  choice  Pq  =  =  1/2,  which  implies  =  l/2  as  well. 

Substituting  these  values  in  (6)  and  using  p  *  1  -  a,  we  find 

c(Cgyin)  =  1  +  a  log  a  +  p  log  p 
for  the  capacity  of  the  BSC. 

The  fundamental  importance  of  the  channel  capacity  as  an  information- 
theoretic  quantity  stems  from  the  role  that  it  plays  in  Shannon's  second 
coding  theorem  (the  so-called  noi sy  coding  theorem),  which  it  is  safe  to  say 
contains  most  of  the  substantive  content  and  technical  promise  of  information 
theory  (taken  together  with  corresponding  studies  of  finite-length  block  coding 
like  [l]).  This  theorem  asserts  that  with  proper  encoding  it  is  possible  to 
transmit  information  at  any  rate  less  than  capacity  with  arbitrarily  small 
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probability  of  error,  provided  that  the  block  length  of  the  code  is  long  enough, 
and  furthermore  that  regardless  of  the  encoding  scheme,  errors  will  always  be  made 
if  one  attempts  to  transmit  information  at  a  rate  greater  than  capacity.  Much 
space  has  been  devoted  in  the  literature  to  a  rigorous  demonstration  of  this 
theorem  for  an  appropriately  large  class  of  sources  and  channels,  and  the  whole 
subject  is  a  difficult  one  which  we  shall  not  go  into  lie  re.  The  interested 
reader  is  referred  to  the  papers  of  Khinchin  [3]  and  the  book  by  Feinstein  [7]  j 
the  latter  author  played  an  important  role  in  developing  a  rigorous  proof  of 
the  noisy  coding  theorem. 


We  turn  now  to  the  question  of  how  the  mathematical  operation  symbolized 

by  (7)  is  to  be  carried  out  in  general.  This  operation  involves  more  than  a 

— > 

simple  maximization  problem,  since  the  vector  P  in  question  is  subject  to  the 
constraint  that  it  be  a  vector  with  non-negative  canponents  which  add  up  to 
unity.  We  begin  by  writing  (3)  in  a  form  which  explicitly  exhibits  its 
dependence  on  the  input  probabilities  P^j  this  form  is 


(8) 


->  ,  m  n 

R(P)  «  -  <  log  P^>+  <  log  Pi;)>  —  ^  T^PjPj^og 


m 


m  n 


pipijloepir 


To  incorporate  the  constraint 
m 


we  add 


5 


For  the  time  being  we  neglect  the  additional  constraint  that  P^  >  0 
which  will  be  discussed  below  in  connection  with  Muroga's  work. 


1  <  i  <  m 


j 
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to  (8),  where  X  is  an  undetermined  Lagrange  multiplier.  Differentiating  the 
sum  of  these  two  terns  with  respect  to  P^,  1  <  i  <  m  ,  and  equating  the  result 


to  zero,  we  obtain 
n 


(9) 


^Pi3—  m 


log 


Lii. 


M- 


1  <  i  <  m  , 


where  (j,  **  X  -  (l/log  e)  is  a  new  constant.  Multiplying  (9)  by  and  summing 
over  i,  we  find  that 


H  -  c  » 


where  c  is  the  capacity  of  the  channel  C. 

Suppose  now  that  the  channel  C  under  consideration,  with  channel  matrix 
(2),  is  square  (rc  *  n)  and  that  det(C),  the  determinant  of  (2),  is  non-vanishing. 
Then  the  inverse  matrix  C~\  satisfying  the  matrix  equation 


(30) 


ccf1  =■  C^C  «  I 


(where  I  denotes  the  unit  matrix)  exists.  If  we  denote  the  elements  of  C 
by  pT^  ,  1  <  i,  j  <  m  ,  then  (ID)  reads 


-1 


m  i  m  i 

ZZ  PijPjk  “^P"<  ’  6< 


j-1 


■ij  *jk  vik  * 


1  <  i,k  <  m  , 


where  6^  is  the  Krone  eke  r  delta  symbol,  equal  to  1  when  i  =  k  and  0  when 
i  ^  k.  Multiplying  (?)  by  p^  and  summing  over  i,  we  obtain 
m  m  n  m  1,1  1 

g ^ pikpij loe pij - ^  pik  >  1^k;m  • 
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Hence 


m  Bis,,  ,  m  a  ,  ) 

g  PiPjk  ■  g  ^  1&P13  log  p13|  ,  1  i  k  < 


m 


where  by  exp(x)  we  mean  2X,  the  base  appropriate  to  our  choice  of  unite. 
Multiplying  by  p£^  and  summing  over  k,  we  obtain 


(11)  pi  ‘  S  ^ exp  {-CS  *  *  £  £  log  p«j 

Finally,  following  Muroga  [8] ,  we  note  that 

m  ,  a  t  m  m 

(12>  • 

so  that  (11)  can  be  simplified  to 


m  ^  ®  1 


,  1  <  i  <  m  . 


(13)  P±  ■  g  P^  .=5-  |-c  -  g  ^  P^P13  log  Pi;)  j  , 

Thus,  the  channel  capacity  is  the  number  c  which  when  substituted  in  (13) 
gives  Pi  >  0  and 

m 


Then  with  this  value  of  c,  (13)  gives  the  corresponding  rate-maximizing 
values  of  the  input  probabilities  P^,  1  <  i  <  m  .  Explicitly,  we  form  the 
sum 


m 


and  use  the  relation  (12)  again,  obtaining 
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m  m 


n  m 


m  n 


•  £ -  1-  *5^ « * 108  M  • 


Solving  for  c,  we  get 


(111)  c  ■  log 


whore  it  will  be  recalled  that  both  log  and  exp  are  taken  to  the  base  2. 


Computations  are  simplified  by  defining  (after  Muroga  [8])  the  auxiliary 


vector 


->  2 
X  =  . 


which  satisfies  the  matrix  equation 


->  -> 

C  X  -  -  H  , 


where  H  I3  the  row-entropy  vector  of  the  channel  C,  i.e.  the  vector 


PUl0g 
Pjjlog  P2i 


p  .log  p  . 
Mtu.  6  *nu. 


-  lit  - 


"V  — s> 

It  follows  from  (15)  and  (16)  that  X  *  -C-"1  H  ,  i.e.  that 


m  -1  m 


Xl  '  ^  PJk  108  *  1^i2”  * 

— ■> 

so  that  in  terns  of  the  components  of  X  ,  (lU)  becomes  simply 

m 

(17)  c  =  log  ~  exp 

Moreover  (13)  becomes 

m  , 

Pj.  *  exp(-c)  %  exp  Xk  ,  1  <  i  < 


m 


Since 


=  cof(p5Jc)/det(C)  , 

where  cof(p^)  is  the  cofactor  of  the  element  p^>  we  finally  have 


(18) 


p.  .  aw  a.t 

i  det(C) 


Pll*.. 

exp(Xn ) 

....  exp(X  ) 
m 

Pirn- 

nnm 

where  the  matrix  in  (18)  differs  from  the  channel  matrix  C  by  having  the 
entries  exp(X1),  exp(Xm)  instead  of  p^,  . ..,  in  the  i’th  row. 


The  method  just  described  requires  modification  if  it  leads  to  negative 
input  probabilities,  and  more  generally  if  det(C)  *  0  or  the  sizes  of  the  input 
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and  output  alphabets  are  different.  Details  of  how  to  deal  with  these  various 
cases  are  given  in  Muroga's  paper  [8],  to  which  the  interested  reader  is 

r 

referred.  There  is  also  available  a  different  and  more  easily  visualized 
approach  to  the  general  problem  of  capacity  due  to  Shannon  [9]. 

We  conclude  our  present  discussion  by  discussing  a  one-parameter 
family  of  ternary  channels  which  for  suitable  values  of  the  parameter 
leads  to  a  negative  probability  for  one  of  the  input  symbols,  which  must 
therefore  then  be  suppressed  according  to  Muroga.  Consider  the  channel 


is 

1 

‘  So 

TC&V 

1 

IS 

The  row-entropy  vector  is 

H 
1 
H 


1 

2 


l-o 

0 

1-a 


1 

I 


where  0  <  a  <  1.  The  inverse  matrix 

1 


,-l 


2o 

1 


2  (1-a/ 
1 

"  IS 


a 

5—1 


H  « 
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— > 

and  the  vector  X  is 


— > 

X  - 


-1 


H-a 


where 


H  =  -a  log  a  -  (1  -a)  log  (l  -  a  )  . 


Using  (17)  and  (1.8)  we  find  that  the  symbol  corresponding  to  the  middle  row  of 
C  should  be  used  with  the  probability 


1  +  exp  (~  fS) 
1  +  exp  Sr) 


if  capacity  is  to  be  achieved.  This  symbol  should  be  suppressed  when  the 
numerator  goes  negative.  This  happens  when  a  >  a  ,  where  is  the  solution 
of 


or 


(19) 


log  a  =  -  a 
6  0  o 


\\ 


The  solution  of  the  transcendental  equation  (19)  is  a.Q~  0.6m.  When  p  >  0, 
the  capacity  of  the  channel  is 


log 


bits/second  , 
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whereas  when  p  <  0,  the  channel  matrix  reduces  to 

a  1-a  0 
0  1-a  a 


This  is  the  matrix  of  the  binary  erasure  channel  (BBC),  i.e.  the  channel  in 
which  C*s  and  I’s  are  transmitted  and  0's,  l’s  and  x's  are  received!  o  is  the 
probability  that  a  0  or  1  is  received  correctly  and  1-a  the  probability  that 
a  0  or  1  is  received  as  an  x.  Since  received  0's  and  I's  are  always  correct, 
idle  re  as  a  received  x  is  always  wrong  and  equally  likely  to  have  come  from  a 
0  or  a  1,  the  capacity  of  the  BEC  is  obviously  a  bits/second.  It  is  interesting 
to  note  that  the  capacity  of  C  is  1  bit/second  both  when  a  *  0  and  a  *  1,  but 
when  a  -  0  the  symbol  corresponding  to  the  middle  row  should  be  sent,  whereas 
when  a  ■  1  it  should  be  suppressed. 


3>.  The  general  binary  channel 

Using  Muroga's  method,  Silverman  [lO]  has  made  a  detailed  study  of  the 
general  binary  channel  C(a,p)  defined  by  ($)  •  We  sketch  without  proof  some 
of  the  results  obtained  in  his  paper: 

l)  The  capacity  c(a,p)  of  the  general  binary  channel  is  given  by  the 


formula 

-0H(e)  +  cH(p) 

c(a,p)  - - +  log 

0  -  a 

The  function  c(a,p)  has  the  symmetries 


1  +  exp 


H(a)  -  H(p) 


\ 


p.a  )  J 


c(a,p)  -  c(p,a)  -  c(l  -  a,  1  -p)  •  c(l  -  p,  1  -  a)  , 


*Loeb  [ll]  has  also  studied  some  aspects  of  the  general  binary  channel. 


and  defines  a  surface  over  the  unit  square  0  <  a  ,  p  <  1.  Lines  of  constant 
c(o,p)  are  shown  in  Fig.  1 

2)  Capacity  is  achieved  if  O’ a  are  transmitted  with  probability 


*>aP> 


P(M)'1  -  (M)"1 


1  +  exp 


H(p)  -  H(o)N 

P  -  « 


-1 


PQ(ajp)  satisfies  the  relation 


<  PQ(a,p)  <  1  -  i  ~  0.63  , 


which  explains  the  footnote  in  Section  1«  The  function  Po(ot,p)  is  discontinuous 
at  the  points  a  *  p  »  0  and  a  *  p  ■  lj  it  has  "the  symmetries 

pQ(a,P>  “  p0<l-*»  1-P)  -  1  -  P0(P,a)  -  1  -  P0(l-P,  l-o )  . 


Lines  of  constant  PQ(a,p)  are  shown  in  Fig.  2. 

3)  At  capacity,  O's  are  received  with  probability 


I,  . 

/h(P)  -  H(a)\ 

p0(“»p)  - 

1  +  exp  - 

\  P  -  «  / 

L  \  /  J 

The  function  P^(a,p)  has  the  symnetries 

p’(o»P)  -  Po(p,«)  -  1  -  P^l-a,  1-P)  -  1  -  P*(l-p,  l-o)  . 


Lines  of  constant  P^(a,p)  are  shown  in  Fig.  3» 

The  reader  interested  in  other  properties  of  the  general  binary  channel, 
e.g.,  the  form  of  the  channels  giving  the  maximally  asymmetric  input  probability 
distributions  (PQ''N-'  l/e  or  P  ^1  -  1/e),  the  probability  of  error  for  the 
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Fig.  1  -  lines  of  constant  c(o,p). 
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general  binary  channel,  etc.,  is  referred  to  Silverman's  paper  [lo], 

6.  Channels  with  fading 

The  communication  situation  dealt  with  so  far  can  be  indicated  schema¬ 
tically  by  the  diagram 


T  ->  C  ->  R  , 

i.e.,  information  is  sent  from  a  transmitter  T  to  a  receiver  R  through  a  channel 
C,  characterized  by  a  stochastic  matrix  (see  Section  3) •  (We  make  no  distinction 
between  the  primary  information  source  S  of  Section  2  and  the  transmitter  T, 
although  in  general  there  is  the  problem  of  (noiselessly)  encoding  the  output 
of  S  so  as  to  match  the  input  of  T.)  We  now  consider  the  following  generali¬ 
zation  of  this  situation!  Instead  of  one  (memoryless)  channel  C,  let  there  be 
a  family  of  (raemoryless)  channels  Ca,  where  the  index  a  ranges  from  1  to  s, 
and  let  the  channel  which  is  actually  present  at  a  given  time  of  transmission 
depend  on  the  state  of  a  random  device  N  (N  for  "  nature").  Schematically,  we 
have 

N 

I 

T  -*C  — >  R  , 

i.e,,  information  is  sent  from  the  transmitter  T  to  the  receiver  R  through  the 
channel  C,  the  state  of  which  depends  on  the  state  of  nature  N.  We  assume 
that  the  state  a,  1  <  a  <  a  ,  chosen  by  nature  is  statistically  independent 
of  past  and  future  states  of  N  and  of  the  transmitted  sequences  as  well;  let 
p  be  the  probability  that  nature  chooses  the  state  a.  The  channel  C  is  now 
represented  by  the  family  of  stochastic  matrices 


V 
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1  <  a  <  s  . 


Since  the  transition  probabilities  p^j(a)  are  now  random  variables,  a  new 
element  of  randomness  has  entered  into  the  problem.  Such  a  model  might  be 
used  to  give  an  abstract  representation  of  communication  In  the  presence  of 
»»  fading"  ,  and  with  this  in  mind  we  refer  to  the  totality  of  channels  0o, 

1  <  °  <  s,  as  a  channel  with  fading. 

A  natural  problem  in  the  theory  of  channels  with  fading  is  that  of 
finding  the  channel  capacity  for  various  conditions  of  knowledge  of  N  at 
the  transmitter  and  receiver.  There  are  four  possible  cases,  which  can  be 
schematically  represented  as  follows: 


Case  1. 


Case  2. 


Case  3* 


N 

1 

T  — *  C  — ♦  R 


K 

T  — ►  C  — ♦  R 
N 

^  1  N 

T  — »  C  — *  R 
H 

^  i 

T  — »  C  — > 


Case  U 


R 
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Case  1,  which  we  have  already  encountered,  represents  the  situation  in  which 
neither  the  transmitter  nor  the  receiver  knows  nature's  state,  so  that  N  is 
effectively  just  more  noise  in  addition  to  that  already  included  in  C.  Case 
2  is  the  situation  in  which  the  receiver  but  not  the  transmitter  knows  nature's 
state.  Case  3  is  the  situation  in  which  both  the  transmitter  and  the  receiver 
know  nature's  state,  and  Case  U  is  the  situation  in  which  the  transmitter  but 
not  the  receiver  knows  nature's  state.  We  now  give  expressions  for  the 
capacity  in  all  four  cases. 

let  p(a,i,j)  be  the  joint  probability  that  nature  chooses  the  state  a,- that 
x^  is  transmitted  and  that  y.j  is  received.  In  cases  1  and  2,  where  the 
transmitter  is  ignorant  of  nature's  state,  we  have 

(20)  p(a,i,j)  «  PaPiPij(a) 

In  case  3,  where  the  transmitter's  action  can  depend  on  nature's  state,  we  have 
(a.)  p(a,l,J)  -  PaP1(aPy{e) 

where  P.  is  the  probability  of  choosing  x.,, given  that  nature's  state  is  o. 
(Case  U  requires  special  treatment j  see  below.)  Generalizing  the  angular 
bracket  notation  of  Section  3,  we  write 

<C. f(o,i, j)^>*  • 

We  now  derive  expressions  for  the  channel  capacity  in  the  four  different  cases. 


Case  1.  Since  neither  the  transmitter  nor  the  receiver  knows  nature's. 
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state,  the  channel  has  the  same  capacity  as  the  average  channel 


'  ■  t:  w* 

c«l 


with  transition  probabilities 


£  ** 


(o)  . 


Explicitly  we  have,  from  (8) 


C1  ‘ 


Max  pij 

->  log  - 


for  the  capacity  in  this  case. 


Case  2.  Since  now  the  receiver  can  use  its  knowledge  of  the  pair 
(a,y^)  to  infer  x^  we  have,  from  (3) 

Max  /  p(a,i,3)  x. 

(22)  c,  -  <T  log  - - - “  > 

2  P  \  P-p(o,j)  ^ 


for  the  capacity  in  this  case,  where  p(a, j)  is  the  probability  of  the  pair 

(a>7.j)» 


P(«»3)  p(°>i»3)  . 


Using  (20)  we  can  rewrite  (22)  as 


26  - 


(23) 


p13(a) 


£ 


— >  -> 

Let  R  (P)  be  the  rate  in  the  channel  C  when  using  the  Input  rector  P  •  Then 
<x  a 

(23)  is  just 


(2k) 


°2 


Max 

P 


In  other  words,  the  capacity  in  case  2  is  the  maximum  average  rate  when 

•“%> 

driving  all  the  channels  CQ  with  the  same  input  probability  vector  P.  Clearly, 
c2  >  ci  ,  since  we  are  now  using  more  detailed  knowledge  of  the  channel. 

Case  3.  Now  the  transmitter  can  base  its  choice  of  input  probabilities 
on  nature's  state,  choosing  a  suitable  input  vector 


for  each  state,  1  <  o  <  Moreover,  as  in  case  2,  the  receiver  has  the  pair 
(a,y^)  available  to  infer  x^.  Therefore  we  have 


Max 


p(«>i* i) 


(2*) 


log 


pi,«P(a^> 
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for  the  capacity  in  this  case,  where  the  maximization  is  over  all  Pa  , 
1  <  a  <  a  *  Using  (21),  we  can  rewrite  ( 2 $)  as 


In  other  words,  the  capacity  in  case  3  is  the  maximum  average  rate  when 
driving  each  channel  with  its  own  input  probability  vector  Pfl  .  It  follows 
at  once  that 


(26)  c3-gpoc(Ca)  , 


i,e.,  the  capacity  is  just  the  average  capacity  of  the  channels  , 

1  <  a  <  s.  Clearly  c^  >  c^  ,  since  now  the  transmitter  as  well  as  the 
receiver  is  using  the  extra  information  about  the  channel . 

Case  U  is  more  complicated.  In  this  case  Shannon  [12]  has  shown  that 
the  capacity  of  the  channel  with  fading  is  the  same  as  the  capacity  of  a  new 
m8  x  n  (non-fading)  channel  C1,  with  input  letters  consisting  of  the  m8 

<  si  and  output  letters 
consisting  of  the  n  letters  y^,  1  <  j  <  n,  where  the  transition  probabilities 
for  C*  are  defined  by  - 

a 

i  *  ^  PaPi  j(<*)  • 

Thus  the  capacity  in  this  case  is 

c^  ■  c(C*)  , 


s-tuples  (xi  ,  x ^  x^  ),  1  <  Ijt  ig,  i 


and  clearly  <  c^  <  *  We  now  give  two  examples  which  illustrate  the 

various  cases. 

Example  1.  Suppose  nature  has  two  states  1  and  2  with  probabilities 
o  and  p,  respectively^  and  suppose 

* 

Then  the  average  channel  £  is  the 
C  ■  aC^  +  pCg  ■ 

so  that  -  1  +  o  log  a  ♦  p  log  p  bits /second  (see  Section  U).  If  nature's 
state  is  known  by  the  receiver  or  by  both  the  transmitter  and  the  receiver, 
then  the  capacity  of  the  fading  channel  is  obviously  1  bit/second,  achieved 
by  transmitting  0's  and  l's  with  equal  probability  and  interchanging  O's  and 
l's  at  the  receiver  when  the  channel  is  in  state  2.  Bqs.  (2h)  and  (26)  confirm 
that  Cg  a  =  1.  When  only  the  transmitter  knows  nature's  state,  the 
capacity  is  again  1  bit/second,  achieved  by  transmitting  O's  and  l's  with 
equal  probability  and  interchanging  O's  and  l's  at  the  transmitter  when  the 
channel  is  in  state  2.  In  this  instance  (case  k )  Shannon's  construction  asserts 
that  c ^  *  c(c’),  where 


1  0 
0  1 


°2  * 


0  1 
1  0 


sC,  i.e. 


c  p 
P  a 
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Denoting  the  rows  of  this  matrix  by  the  vectors  A^,  (e.g*  A^  »  (q,p)  ), 

we  have 


— >  ->  -> 

A^  -  aAg  +  pA3 


— >  — >  —o 

Au  "  PA2  +  ^3 


It  follows  by  an  argument  due  to  Shannon  [9]  that  the  symbols  corresponding 
to  the  first  and  fourth  rows  of  C '  have  to  be  suppressed  if  capacity  is  to  be 
achieved.  Dropping  the  first  and  fourth  rows  of  C',  we  get  the  matrix  , 
so  that  *  c(C^)  *  1  bit/second,  as  required. 

Example  2.  Let  there  be  three  states  with  equal  probability,  and  let 


1  0 

1  1 
2  Z 


°2  * 


1  1 
2  2 
1  1 
2  1 


1  i 

2  2 

0  1 


Then  if  neither  the  transmitter  nor  the  receiver  knows  nature’s  state,  the 
capacity  of  the  fading  channel  is  that  of  the  average  channel 


Z 


2  1 

3  3 

1  2 

?  Z 


* 


l.e.  ^  -  c(S)  -  1  +  (1/3)  log  (1/3)  +  (2/3)  log  (2/3)  -  (5/3)  -  log  3  ~.082 
bits/seoond.  If  only  the  receiver  knows  nature’s  state,  then  by  (21*) 


Max 


k  k<« 


♦  R2(P) 


E3(P)] 


30 


By  symmetry,  the  maximum  is  achieved  for 

1 
2 

j 

1 
2 


and  an  elementary  calculation  shows  t-hat-Cg- »  1  -  (l/2)  log  3  ~  .208 
bits/second  >  .  If  both  the  transmitter  and  the  receiver  know  nature's 

state,  then  by  (26)  - 


c3  ■  f  c^Cl)  » 
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By  a  simple  extension  of  the  argument  given  in  connection  with  the  preceding 
example ,  we  have 

c^  *  c(Cr )  ■  c(S)  ■  c^  ~  ,082  bits/second  , 


so  that  in  this  case,  unlike  example  1,  the  transmitter  cannot  use  its 
knowledge  of  nature's  state  to  increase  the  channel  capacity. 


7.  Cascaded  channels 

Let  C  be  an  i  *  n  channel  matrix  with  output  alphabet  y^,  y^  ,  and 
let  c'  be  ah  m  x  n  channel  matrix  with  input  alphabet  y^,  yffl  (i.e, 

identical  to  the  output  alphabet  of  C).  Then  we  agree  to  apply  y^  to  the 
input  of  C '  whenever  y^  is  received  at  the  output  of  C;  this  mode  of  channel 
combination  is  called  cascading.  Denote  by  p^  and  p^  the  elements  of  the 
matrices  C  and  c',  and  denote  by  p”^  the  elements  of  the  matrix  c"  obtained, 
by  cascading  C  and  C*,  as  just  described.  Then,  since  obviously 


ti 


C  is  obtained  from  C  and  C  by  matrix  multiplication,  i.e.  C 


CC 


Since 


matrix  multiplication  is  in  general  non-commutative,  the  same  is  true  of 


channel  cascading. 

The  problem  of  cascading  identical  binary  channels  C(a,p) 
especially  simple,  in  view  of  the  identity 


a  1-a 

P  1-P 


is 


(27)  Cn(a,p)  « 


a(n)  l-a(n) 

i  / 

p  1-a 

a-1  1-a 

1 

p(n)  1-p(n) 

■  \  ’ 
1-a+p  ^ 

p  1-a 

-  (a-p)n 

p  -p 

32  - 


which  is  easily  derived  from  a  general  formula  of  matrix  algebra  (see  e.g.  [13]). 
(We  exclude  the  trivial  cases  a-1,  p«0  and  a»0,  p»l,  corresponding  to  the 


noiseless  channels 
be  derived  from  (27): 


1  0 
0  1 


and 


0  1 
1  0 


.)  A  number  of  simple  consequences  can 


1)  Since  |a  -  p|  <  1,  we  have 

(28)  Gn(o,p)  !C(®)(t,p) 


1 

p  1-a 

■  — . — 

1-a+p 

p  1-a 

m 

a(to)  l«x(co> 

p(m)  n>(co11 


(OO) 

The  limiting  channel  C'  '(a,p)  obviously  has  zero  capacity,  in  accord  with 
the  intuitive  idea  that  an  infinite  cascade  of  noisy  channels  must  destroy 
any  information  fed  into  it. 

2)  Since 

4(°0)  p(n)_  p(®)  f 

I-/00*  '.W-.t®1  "a-1  ’ 


all  the  channels  Cn(a,p)  lie  on  a  straight  line  in  the  (a,p)  square  passing 
through  the  point  corresponding  to  the  limiting  channel  Cv  ;(a,p)  and  the 
point  (0,1).  Thus,  referring  to  Fig.  1,  we  have  the  following  simple  interpre¬ 
tation  of  the  operation  of  cascading  a  binary  channel  with  itself:  Using  (28), 
draw  the  straight  line  just  described;  then  as  n  increases,  Cn(a,p)  approaches 
Cv  ;(a,p)  along  this  straight  line,  moving  alternately  from  one  side  of  the 
zero-capacity  line  a  ■  p  to  the  other  if  a  -  p  <  0  . 


The  operation  of  channel  cascading,  or  equivalently  of  multiplying 
channel  matrices,  is  a  partial  ordering  in  the  following  sense:  Given  any  two 
channels  and  C2»  then  is  said  to  include  Cg,  written  =s  ^2  *  ^  *k®re 
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exists  a  channel  C  such  that  C^C  *  Cg  ,  i.e.  if  Gg  can  be  obtained  from 
by  cascading.  Channel  inclusion  has  the  defining  properties  of  a  partial 
ordering,  namely 

1)  C  =  C  ,  for  any  C  j 

2)  =  Cg  ,  Cg  =  Cy  implies  =  C^  . 

On  the  other  hand,  given  arbitrary  channels  and  Gg  ,  neither  of  the 
relations  o  Cg  ,  Cg  ^  may  hold,  i.e.  and  Cg  may  not  be  comparable  . 

In  the  case  of  binary  channels,  the  structure  of  the  partial  ordering 
is  very  simply  displayed.  Suppose  we  have  two  binary  channels 


a  1-a 

a*  1-a* 

c  - 

P  1-P 

,  C1  - 

P*  1-P' 

and  assume,  as  we  can  without  loss  of  generality,  that  {5  <  a,  p'  <  o' .  Then, 
following  a  procedure  suggested  by  Birnbaum  (private  communication),  we  first 
prove  the  following  lemma: 

A  necessary  and  sufficient  condition  for  C  ^  C  *  is  that  the  interval 
(Lj,Lg)  contain  the  interval  (L^,  Lg),  where 

\  -  p/a  ,  -  p'/a'  ,  Lg  -  (l-p)/(l-a)  ,  Lj  -  (1-p' )/(l-a‘)  . 

(Note  that  in  our  case  0<L^<l<Lg<  oo  ,  0<L^<l<Lg<oo.) 

To  show  the  necessity,  we  suppose  that 

X  1-x 
C u  *  y  1-y 

#If  c(C)  denotes  the  capacity  of  C,  then  c(C)  >  c(C')  is  a  necessary  but  not 
sufficient  condition  for  C  s  C ’ .  ” 
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is  the  channel  such  that  CCM  ■  C* .  Hien,  doing  the  matrix  multiplication 


explicitly,  we  find  that 

L.'  -  p’/o*  *  - >  p/a  ■  L, 

A  ax+(l-*)v  *“ 


since  y  >  0,  and 

■» 

since  x  <  1. 

To  show  the  sufficiency,  we  solve  formally  for  the  parameters  x,y  of 


l ’  -  (i-p'j/d-o')  -  <  (i-p)/(i-o)  .  % 

1  1  -  ax  -  (l-«)y 


finding 


ap1  -s'p 


a'  -p' 

x°-aft“+y  • 

Since  a  *-  P '  >  0,  a  -  p  >  0  ,  and  ap '  -  a  'p  >0  by  the  hypothesis  that  , 

we  see  that  0  <  y  <  x.  Moreover  y  <  x  <  1,  since  ap1-  a'p  +  a'-0  <  a  -  p 
is  an  easy  consequence  of  the  hypothesis  <  L^.  This  completes  the  proof 
that  C  3  C*  and  (L^Lg)  =>  (3L^,L^)  are  equivalent  statements. 

Using  the  lemma,  we  can  give  a  simple  geometrical  model  of  the  partial 
ordering  of  binary  channels  under  cascading.  Plot  the  point  (a,p)  corresponding 
to  C  in  the  unit  square,  and  draw  the  straight  lines  from  the  point  (a,p) 
to  the  points  (0,0)  and  (1,1),  as  shown  in  Fig.  U.  Then  all  the  points  in 
the  region  shaded  with  vertical  lines  represent  channels  contained  in  C, 
i.e.  channels  which  can  be  obtained  from  C  by  cascading.  This  follows  by 
applying  the  lemma  in  the  part  of  the  region  lying  below  the  line  a*p,  and 

^  changes  a  channel  into  its  reflection 


then  noting  that  multiplication  by 


1  0 


in  the  line  a-p.  Moreover,  it  follows  from  this  construction  that  all  points 
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in  the  region  shaded  by  horizontal  lines  represent  channels  containing  C. 
Similarly,  points  lying  outside  of  the  shaded  regions  represent  channels 
which  are  not  comparable  with  C,  i.e.  which  cannot  be  reached  from  G  either 
by  prenmltiplication  or  postmultipli cation  by  any  binary  channel. 


Fig.  li  -  niustr:  :ing  the  partial  ordering  of  binary  channels. 

For  further  discussion  of  topics  related  to  channel  cascading,  see 
Shannon  [lU] ,  Bimbaum  [ijQ  and  Silverman  [io] . 

8.  Charnels  with  memory 

We  now  consider  a  simple  idealized  communication  system  suggested  by 
Chang  [l6],  which  leads  naturally  to  the  study  of  channels  with  memory. 

Let  the  channel  input  be  a  source  emitting  binary  digits,  and  let  the  channel 
be  such  that  0*s  and  l's  emitted  at  the  time  t  *  0  are  represented  by  the 
wavefonae 

f0(t)  -  A  exp  (-t/tf)  , 


fj{t)  -  -A  exp  (-t/'t  ) 
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respectively,  in  the  absence  of  noise*.  The  effect  of  noise  will  be  repre- 
sented  by  the  addition  of  white  Gaussian  noise  of  r.m.s.  voltage  o'  . 

Suppose,  to  consider  the  simplest  non-trivial  case,  that  the  source  produces 
the  digits  in  Isolated  doublets,  i.e.,  let  a  pair  of  binary  digits  separated 
by  an  amount  y  ^  be  transmitted,  and  then  let  there  be  a  pause  in 
transmission  long  enough  to  allow  substantially  complete  decay  of  the 
exponential  exp(~t/c').  Finally,  suppose  that  we  use  synchronous,  threshold 
detection  at  the  receiver,  i.e.,  at  the  times  tQ  and  tQ  +  y  corresponding  to 
transmission  of  a  doublet,  we  interpret  a  positive  voltage  as  a  0  and  a 
negative  voltage  as  a  1. 

We  now  proceed  to  find  the  capacity  of  this  simple  channel.  At  a  time 

t  such  that  the  channel  has  recovered  from  the  effects  of  previous  transmitted 
o 

signals  the  channel  is  described  by  the  matrix 

a  l-a 

C  - 

P  1-p 

where 

a  -  Prob  [fQ(0)  +  K  >  0\  , 
p  -  Prob  [f  (0)  +  K  >  0]  , 

2 

and  ?  is  a  Gaussian  random  variable  with  mean  0  and  variance  o'  .  (Specifically, 

We  have  in  mind,  for  example,  a  situation  where  O’ a  and  l’s  are  encoded  into 
sharp  positive  and  negative  pulses  of  amplitude  A  and  then  sent  through  a 
channel  whose  transmission  characteristics  resemble  those  ofanRC  filter 
with  time  constant  'b  . 

By  H  white”  noise,  we  mean  noise  with  a  constant  power  spectral  density,  or 
equivalently  with  a  correlation  function  which  is  a  delta  function.  Gaussian 
white  noise  has  no  memory.  This  is,  of  course,  a  limiting  case,  approached 
only  when  the  noise  band  width  is  much  greater  than  that  of  the  signals. 


a  is  the  probability  that  a  0  transmitted  at  time  tQ  is  interpreted  as  a  0, 
and  p  is  the  probability  that  a  1  transmitted  at  time  tQ  is  interpreted  as  a  0. 
Similarly,  aQ,  pQ  and  o^,  p1  are  the  corresponding  probabilities  at  time 
tQ  +  y>  under  the  hypothesis  that  a  0  or  a  1  was  transmitted  at  time  tQ, 
respectively  (see  below)  ) .  Clearly  we  have 

a  -  Prob  (K  >  -A)  -  1  -  F(-A)  , 
p  -  Prob  U  >  A)  -  1  -  F(A)  , 

where 


is  the  distribution  function  of  the  random  variable  £.  It  follows  from  the 
symmetry  of  F(x)  that  a  +  p  »  1,  i.e.  that  C  is  a  symmetric  channel.  At  the 
time  t  ♦  Y  *  there  are  two  possibilities.  If  a  0  was  transmitted  at  time 
tQ,  then  the  channel  is  described  by  the  matrix 

“o  i-o 
Po  1_Po 

where 


oQ  -  Prob  [fQ(0)  ♦  fQ(r)  ♦  K  >  0]  -  1  -  F[^A(l+p)]  , 

P0  -  Prob  [f0(0)  +  fx(Y)  +  K  >  0]  -  1  -  F[-A(l-p)]  , 

and  p  ■  exp(^f/v).  Clearly  we  have  aQ  +  pQ  >  1  and  CQ  is  asymmetric. 

Similarly,  if  a  1  was  transmitted  at  tQ,  then  at  time  tQ  +  Y  the  channel  is 
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described  by  the  matrix 

al  X^1 

c!- 

where 

#3.  -  Prob  [fl(0)  +  fo(Y)  ♦  «  >  o]  -  1  -  P[A(l-p)]  , 
px  -  Prob  [^(o)  ♦  ^(r)  +  s  >  o]  -  i  -  fCa(i+p)]  . 

This  time  <  1  and  again  is  asymmetric. 

To  find  the  capacity  of  the  channel  with  memory,  it  is  simplest  to 
proceed  on  a  doublet  basis  and  enlarge  the  channel  to  a  new  k  x  U  memoryless 
channel  C* ,  with  inputs  and  outputs  consisting  of  the  four  pairs  00,  01,  10 
and  11.  Thus,  we  consider  the  li  x  |  channel  matrix 

.C0  (l-a)e0 

P0X  (l-fiJCj 

whose  elements  are  themselves  2x2  matrices.  Following  Chang  [l6] ,  we 

find  the  capacity  of  0*  by  using  Muroga’s  method  (see  Section  li).  The 
—>  , 

row-entropy  vector  H'  of  C  is  easily  seen  to  be 


H(o)  ♦  H(a  ) 

— > 

H»  - 

o 

H(a)  ♦  H(p0) 
H(p)  +  H(ax) 
H(p)  +  H(p^) 

m 

H(a)  U  ♦  Hc 

H(P)  V  *  ^ 

where  we  hare  introduced  the  two-dimensional  auxiliary  rectors  XQ,  X^ 

corresponding  to  the  channels  C  ,  C.,  respectively,  i.e.,  satisfying  the 
— >  ,  — >  — >  ,  — >  -o 

equations  Xft-  -C“A  HQ  and  X1  -  -C![  H^.  Then,  if  X  is  the  auxiliary  vector 

corresponding  to  the  channel  C,  i.e,  satisfying  the  equation 


— > 

X  s 

xo 

-  -C"1 

H(c) 

*1 

H(p) 

it  follows  from  (29)  that 


(30) 


—  1 

H  M 
o  o 

i 

1-P  a-1 

— > 

Xo 

X  * 

*1 

*i 

c— p 

-p  a 

— > 

*1 

Eq.  (30)  expresses  the  four-dimensional  auxiliary  vector  X*  of  the  expanded 

,  — >  — > 

U  x  U  channel  C  in  terms  of  the  two-dimensional  auxiliary  vectors  X,  XQ 

t 

and  of  the  binary  channels  C,  CQ  and  C^.  Finally,  the  capacity  of  C  in 

bits/doublet  is  obtained  from  (30)  by  using  Eq.  (l?).  Thus,  the  calculation  of 
— c> 

X  and  c(C  )  is  a  simple  matter,  provided  one  has  available  a  table  of  the 

—> 

values  of  the  auxiliary  vector  X  corresponding  to  the  general  binary  channel. 
Such  a  table  is  given  in  Chang's  paper  [l6],  to  which  the  reader  is  referred 
for  further  details. 


The  simple  example  just  given,  where  isolated  doublets  are  transmitted, 
illustrates  the  general  approach  to  the  problem  of  determining  the  capacity  of 
channels  with  memory  of  the  type  under  consideration.  More  generally,  one  can 


consider  the  case  of  isolated  groups  of  n  equally  spaced  digits,  and  then  the 
case  of  a  channel  driven  at  a  uniform  rate  with  no  pauses  for  recovery  frcm 
the  effects  of  previously  transmitted  signals.  (Of  course,  the  latter  case 
corresponds  to  passing  to  the  limit  n  -*>00  in  the  case  of  transmitting 
isolated  groups  of  n  equally  spaced  digits.)  One  can  also  consider  m  x  ns 
channels  with  memory,  where  m  >  2.  Such  problems  have  been  studied  by  Chang 
and  co-workers  [l6j ,  [17] .  Finally,  it  is  natural  to  study  the  case  where 
the  noise  itself  has  appreciable  memory  or  even  the  case  where  there  is 
statistical  dependence  between  the  noise  and  the  signals.  It  appears  that 
much  remains  to  be  done  along  these  lines. 
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Classified  to  he  sent  toi 
Director 

!!.  S.  Naval  Keenarch  Laboratory 
Attn:  Code  5200 
Washington  25,  D.  C, 

Commanding  Officer  and  Director 

U.  S.  Navy  Underwater  Sound  Laboratory 

Fort  Trumbull,  New  London,  Connecticut 

Chief  of  Naval  Research 
Department  of  tho  Navy 
Washington  25,  D.  C. 

Attn:  Code  1:27 

Commanding  Officer  and  Director 

U.  S.  Navy  Electronic#  Laboratory  (Library) 

San  Diego  52,  California 

Chief,  Bureau  of  Ordnance 
Department  of  tho  Nayy 
Washington  25,  D.  C. 

Attn:  Code  Ad3 

Chief,  Bureau  of  Ordnano. 

Department  of  the  Nayy 
Surface  Guided  Hieeile  Branch 
Washington  25,  D.  C. 

Attn:  Code  ReSl-e 

Chief,  Bureau  of  Ordnance 
Department  of  the  Navy 
Washington  25,  D.  C, 

Attn:  Fire  Control  Brnnoh  { ReSh ) 

Department  of  the  Navy 
Bureau  of  Aeronautic* 

Technical  Date  Divieion,  Code  !|106 
Washington  25,  D,  C. 

Chief,  Bureau  of  Ships 
Department  of  the  Nevy 

Washington' 25,  D,  0. 

Attn:  Cod#  817B 


Commanding  Officer 
G.  S.  Naval  Air  Development  Center 
Johns ville,  Pennsylvania 
Attn:  NADC  Library 

Commander 

U.  S.  Naval  Air  Test  Center 
Patuxent  River,  Maryland 
Attn:  ET"315,  Antenna  Branch 

Director 

Naval  Ordnance  Laboratory 
Corona,  California 

Commanding  Officer 

U.  S.  Naval  Ordnance  Laboratory 

Corona,  California 

Attn:  Mr.  W.  Horer.stein,  Division  72 

Airborne  Instruments  Laboratory,  Inc. 

160  Old  Country  Road 

Mineola,  New  York 

Attn:  Dr.  E.  G.  Fubinl,  Director 

Research  and  Engineering  Division 

Aircom,  Inc* 

35U  Main  Street 
Winthrop,  Mass* 

American  Machine  and  Foundry  Company 
Electronics  Division 
1085  Commonwealth  Avenue 
Boston  15,  Mass. 

Attn:  Mrs.  Rita  Moravcsik,  Librarian 

Andrew  Alford,  Consulting  Engineers 
299  Atlantic  Avenue 
Boston  10,  Mass. 

Avion  Division 
ACF  Industries,  Inc* 

800  No.  Pitt  Street 
Alexandria,  Virginia 
Attn:  Library 

Battelle  Memorial  Institute 
505  King  Avenue 

Attn:  Wayne  E.  Rife,  Project  Leader 
Electrical  Engineering  Division 
Columbua  1,  Ohio 

Bell  Aircraft  Corporation 

Post  Office  Box  One 

Buffalo  5,  New  York 

Attn:  Eunice  P.  Hazelton,  Librarian 

Bell  Telephone  Laboratories,  Inc. 
Vhippany  Laboratory 
Whippany,  New  Jersey 
•Attn:  Technical  Information  Library 

Pacific  Division 

Bendix  Aviation  Corporation 

11600  Sherman  Way 

North  Hollywood,  California 

Engineering  Library 

Attn:  Peggie  Robinson,  Librarian 

Bendix  Radio  Division 
Bendix  Aviation  Corp. 

E.  Joppa  Road 

Towson  U,  Maryland 

Attn:  Dr.  D.  M.  Allison,  Jr. 

Director  Engineering  and  Research 

Boeing  Airplane  Company 

Pilotless  Aircraft  Division 

P.0.  Box  3707 

Seattle  21:,  Washington 

Attn:  R.R.  Barber,  Library  Supervisor 

Boeing  Airplane  Company 

Wichita  Division  Engineering  Library 

Wichita  1,  Kansas 

Attn:  Kenneth  C.  Knight,  Librarian 

Boeing  Airplane  Company 

Seattle  Division 

Seattle  lb,  Washington 

Attn:  E.T.  Allen,  Library  Supervisor 

BJorksten  Research  Labs,  Inc. 

P.  0.  Box  265 

Madison,  Wisconsin 

Attn:  Mrs.  Fern  B.  Korsgard 
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Convair,  A  Division  of  General  Dynamics 
Corp. 

Fort  Worth,  Texas 

Aiitn:  K.O.  Brown,  Division  Research 
Librarian 

Convair,  A  Division  of  General  Dynamics 
Corp. 

Sari  Diego  12,  California 
Attn:  Mrs.  Dora  B.  Burke, 

Engineering  Librarian 

Cornell  Aeronautical  Laboratory,  Inc. 
14:55  Genesee  Street 
Buffalo  21,  New  York 
Attn:  Librarian 

Dalrao  Victor  Company 
A  Divieion  of  Textron,  Inc. 

1515  Industrial  Way 
Belmont,  California 
Attn:  Mary  Ellen  Addems, 

Technical  Librarian 

Dome  and  Margolin,  Inc. 

29  New  York  Avenue 

West bury.  Long  Island,  N.  Y. 

Douglas.  Aircraft  Company,  Inc. 

P.O.  Box  200 

Long  Beach  1,  California 

Attn:  Engineering  Library  (C-250) 

Douglas  Aircraft  Co.,  Inc. 

82?  Lapham  Street- 
El  Segundo,  California 
Attn:  Engineering  Library 

Douglas  Aircraft  Company,  Inc. 

3000  Ocean  Park  Boulevard 
Santa  Monica,  California 
Attn:  P.T.  Cline 

Eq.  Sec.  Reference  Files, 

Eq.  Eng.  A250 

Douglas  Aircraft  Company,  Inc. 

2^00  North  Memorial  Drive 
Tulsa,  Oklahoma 

Attn:  Engineering  Library,  D-250 

Electronics  Communication,  Inc. 

1830  York  Road 
Timonlum,  Maryland 

Emerson  and  Cuming,  Inc. 

869  Washington  Street 
Canton,  Mass. 

Attn:  Mr.  W.  Cuming 

Bnerson  Electric  Kfg.  Co: 

8100  West  Florissant  Avenue 

St.  Louis  21,  Missouri 

Attn:  Mr.  E.R.  Breslin,  Librarian 

Syl vania  Elec.  Prod.  Inc. 

Electronic  Defense  Laboratory 
•P.0.  Box  205  -  (Unci) 

Mountain  View,  California 
Attn:  Library 

Fairchild  Aircraft  Division 
Fairchild  Eng.  and  Airplane  Corp. 
Hagerstown,  Maryland 
Attn:  Library 

Farnsworth  Electronics  Company 
3700  East  Pontiac  Street 
Fort  Wayne  1,  Indiana 
Attn:  Technical  Library 

Federal  Telecommunication  Labs. 

500  Washington  Avenue 
Nut ley  10,  New  Jersey 
Attn:  Technical  Library 

The  Gabriel  Electronic* 

Division  of  the  Gabriel  Company 
235  Creacent  Road 
Needham  Heights  9b,  Mass. 

Attn:  Mr.  Steven  Oalagan 


General  Electric  Advanced  Electronics  Center 
Cornell  University 
Ithaca,  New  York 
Attn:  J#  B.  Ti*ayis 

Oeneral  Electric  Company 
Electronics  Park 
Syracuse,  New  York 

Attn:  Documents  Library,  B.  Fletcher 

Building  3-lb3A 

Genera]  Precision  Laboratory,  Inc. 

63  Bedford  Road 

Pleasantyille,  New  York 

Attn:  Mrs.  Mary  0.  Herbst,  Librarian 


Granger  Associates 

Electronic  Systems 

966  Commercial  Street 

Palo  Alto,  California 

Attn:  John  V.  N.  Granger,  President 

Grumman  Aircraft  Engineering  Corporation 
Bethpage,  Long  Island,  N.  Y. 

Attn:  Mrs.  A.  M.  Cray,  Librarian 

Engineering  Library,  Plant  No.  5 


Hughes  Aircraft  Company 
Florence  and  Teals  Street* 

Culver  City,  California 

Attn:  Dr.  L.C.  Van  Atta,  Associate  Director 
Research  Labs. 


International  Business  Machines  Corp. 

Military  Products  Division 

590  Madison  Avenue 

New  York  33,  New  York 

Attn:  Mr.  C.F.  McElwaln,  General  Manager 

International  Business  Machines  Corp. 
Military  Products  Division 
Owego,  New  York 

Attn:  Mr.  D.  I.  Marr,  Librarian 
Department  1:59 


Dr,  Henry  Jasik,  Consulting  Engineer 
298  Shames  Drive 
Brush  Hollow  Industrial  Park 
Westbury,  New  York 

Electromagnetic  Research  Corporation 
711  lbth  Street,  N«  W. 

Washington  5,  D.  C. 


Goodyear  Aircraft  Corp. 

1210  Massillon  Road 

Akron  15,  Ohio 

Attn:  Library  D/120  Plant  A 


Hycon  Eastern,  Inc. 

75  Cambridge  Parkway 
Cambridge,  Mass. 

Attn:  Mrs.  Lois  Seulowit* 
Technicsl  Librarian 


International  Resistance  Company 
U01  N.  Broad  Street 
Philadelphia  8,  Pa, 

Attn:  Research  Library 

Jansky  and  Bsilsy,  Inc, 

1339  Wisconsin  Avenue,  N.  W, 
Washington  7,  D.  C. 

Attn:  Mr.  Delmer  C.  Ports 


The  Hallicrafters  Company 

14:01  West  5th  Avenue 

Chicago  2b,  Illinois 

Attn:  La Verne  LaOioia,  Librarian 

Hoffman  Laboratories,  Ino. 

3761  South  Hill  Street 
Los  Angeles  7,  California 
Attn:  Engineering  Library 

Hughes  Aircraft  Company 
Antenna  Department 
Microwave  Laboratory 
Building  12,  Room  2617 
Culver  City,  California 
Attn:  M.  D.  Adcock 
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Pickard  and  Burns,  Inc. 

2lj0  Highland  Avenue 
Needham  9lf>  Mass. 

Attn:  Dr.  J.  T.  DeBettencourt 

Polytechnic  Research  and  Development 
Company,'  Inc. 

202  Tillary  Street 
Brooklyn  1,  New  York 
Attn:  Technical  Library 

Radiation  Engineering  Laboratory 
Main  Street 
Maynard,  Mass. 

Attn:  Dr.  John  Ruze 

Radiation,  Inc. 

P.  0.  Drawer  37 
Melbourne,  Florida 

Attn:  Technical  Library,  Mr.  M.L.  Cox 

Radio  Corp.  of  America 

RCA  Laboratories 

Rocky  Point,  New  York 

Attn:  P.  S.  Carter,  Lab.  Library 


Ryan  Aeronautical  Company 
Lindbergh  Field 
San  Diego  12,  California 
Attn:  Librarv  -  unclassified 

Sage  Laboratories 
159  Linden  Street 
Wellesley  81,  Mass, 

Sanders  Associates 
9 5  Canal  Street 
Nashua,  New  Hampshire 
Attn:  N.  R,  Wild,  Library 

Sandia  Corporation,  Sandia  Base 
P.0.  Box  5^00*  Albuquerque,  New  Mexico 
Attn:  Classified  Document  Division 

Sperry  Gyroscope  Company 

Great  Neck,  Long  Island,  New  York 

Attn:  Florence  W.  Turnbull,  Engr.  Librarian 

Stanford  Research  Institute 

Menlo  Park,  California 

Attn:  Library,  Engineering  Division 


Lockheed  Aircraft  Corporation 

255?  N.  Hollywood  Way 

California  Division  Engineering  Library 

Department  72-7 ?>  Plant  A-l,  Bldg.  63-1 

Burbank,  California 

Attn:  N.  C.  .Harnois 

The  Martin  Company 
P.  0.  Box  179 
Denver  1,  Colorado 
Attn:  Mr.  Jack  McCormick 

The  Glenn  L.  Martin  Company 
Baltimore  3,  Maryland 
Attn:  Engineering  Library 
Antenna  Design  Group 

Maryland  Electronic  Manufacturing  Corp. 

5009  Calvert  Road 
College  Park,  Maryland 
Attn:  Mr.  H.  Warren  Cooper 

Mathematical  Reviews 
190  Hope  Street 
Providence  6,  Rhode  island 

The  W.  L.  Maxson  Corporation 
h60  West  3l*th  Street 
New  York,  N.  Y. 

Attn:  Miss  Dorothy  Clark 

McDonnell  Aircraft  Corporation 

Lambert  Saint-Louis  Municipal  Airport 

Box  516,  St.  Louis  3,  Missouri 

Attn:  R.  D.  Detrich,  Engineering  Library 

McMillan  Laboratory,  Inc. 

Brownville  Avenue 

Ipswich,  Massachusetts 

Attn:  Security  Officer,  Document  Room 

Melpar,  Inc.  (2) 

3000  Arlington  Boulevard 
Falls  Church,  Virginia 
Attn:  Engineering  Technical  Library 

Microwave  Development  Laboratory 
90  Broad  Street 
Baboon  Park  57,  Massachusetts 
Attn:  N.  Tucket;  General  Manager 

Microwave  Radiation  Company  Inc. 

19223  South  Hamilton  Street 
Gardena,  California 

Attn:  Mr.  Morris  J.  Ehrlich,  President 

Chance  Vought  Aircraft,  Inc. 

931b  West  Jefferson  Street 
Dallas,  Texas 

Attn:  Mr.  H.  S.  White,  Librarian' 

Northrop  Aircraft,  Inc. 

Hawthorne,  California 

Attn:  Mr.  E.  A.  Freitas,  Library  Dept  J  lk 5 
1^01  E.  Broadway 

Remington  Rand  Univ,  -  Division  of  Sperrv 
Rand  Corporation 
1900  West  Allegheny  Avenue 
Philadelphia  29,  Pennsylvania 
Attn:  Mr.  John  F.  McCarthy 

R  and  D  Sales  and  Contracts 

North  American  Aviation,  Inc. 

12211:  Lakewood  Boulevard 

Downey,  California 

Attn:  Engineering  Library  1:95-11? 

North  American  Aviation,  Inc, 

Ios  Angeles  International  Airport 
Los  Angeles  1:5,  California 
Attn:  Engineering  Technical  File 

Page  Communications  Engineers,  Inc. 

710  Fourteenth  Street,  Northwest 
Washington  5,  D.  0. 

Attn:  Librarian 

Philco  Corporation  Research  Division 
Branch  Library 
1:700  Wissachickon  Avenue 
Philadelphia  kk,  Pa. 

Attn:  Mrs,  Dorothy  S.  Collins 


RCA  Laboratories 
David  Sarnoff  Research  Center 
Princeton,  New  Jersey 
Attn:  Miss  Fern  Cloak,  Librarian 
Research  Library 

Radio  Corporation  of  America 
Defense  Electronic  Predicts 
Building  10,  Floor  7 
Camden  2,  New  Jersey 
Attn:  Mr.  Harold  J.  Schrader 

Staff  Engineer,  Organization 
of  Chief  Technical 
Adffd  nistrator 

The  Ramo-Wooldridge  Corporation 
P.0,  Box  1:51:53  Airport  Station 
Los  Angeles  !i5,  California 
Attn:  Margaret  C.  Whitnah, 

Chief  Librarian 

Hoover  Mic^wave  Co. 

9592  Baltimore  Avenue 
College  Park,  Mar viand 

Director,  USAF  Project  RAND 
Via:  Air  Force  Liaison  Office 
The  Rand  Corporation 
1700  ^in  Street 
Santa  Monica,  California 

Rarrtec  Corporation 

Calabasas,  California 

Attn:  Grace  Keener,  Office  Manager 

Raytheon  Manufacturing  Company 
Missile  Systems  Division 
Bedford,  Mass. 

Attn:  Mr.  Irving  Goldstein 

Raytheon  Manufacturing  Company 
Wayland  Laboratory,  State  Road 
Wayland,  Mass. 

Attn:  Mr.  Robert  Borts 

Raytheon  Manufacturing  Company 
Wayland  Laboratory 
Wayland,  Mass. 

Attn:  Miss  Alice  0.  Anderson, 
Librarian 

Republic  Aviation  Corporation 
Farmlngdale,  Long  Island,  N.  Y. 

Attn:  Engineering  Library 

Thru:  Air  Force  Dlant  Representative 
Republic  Aviation  Corp. 
Famingdale,  Long  Island,  N.Y, 

Rheem  Manufacturing  Company 
9236  East  Hall  Road 
Downey,  California 
Attn:  J.  C.  Joerger 

Trans-Tech,  Inc. 

P.  0.  Box  31:6 
Frederick,  Maryland 


Syl vania  Electric  Products,  Inc. 

.100  First  Avenue 
Waltham  5k,  Mass. 

Attn:  Charles  A.  Thornhill,  Report  Librarian 
Waltham  Laboratories  Library 


Lockheed  Aircraft  Corporation 
Missile  Systems  Division  Research  Library 
Box  50h,  Sunnyvale,  California 
Attn:  Miss  Eva  Lou  Robertson, 

Chief  Librarian 


Commander 

AF  Office  of  Scientific  Research 
Air  Research  and  Development  Command 
lkfch  Street  and  Constitution  Avenue 
Washington,  D.  C# 

Attn:  Mr.  Otting,  SRY 


Westinghouse  Electric  Corp. 
Electronics  Division 
Friendship  Tnt'l  Airport  Box  7k6 
Baltimore  3,  Maryland 
Attn:  Engineering  Library 


Systems  Laboratories  Corporation 
lkfi52  Ventura  Boulevard 
Sherman  Oaks,  California 
Attn:  Donald  L.  Margerum 

TR3,  Inc. 

1?  Union  Square  West 
New  York  3,  N.  Y. 

Attn:  M.  L.  Henderson,  Librarian 

A.  S.  Thomas,  Inc. 
l6l  Devonshire  Street 
Boston  10,  Mass. 

Attn:  A.  S.  Thomas,  President 

Bell  Telephone  Laboratories 
Murray  Hill 
New  Jersey 

Chu  Associates 
p.  0.  Box  387 
Whitcomb  Avenue 
Littleton,  Mass. 

Microwave  Associates,  Inc. 
Burlington,  Maas. 

Raytheon  Manufacturing  Company 
Missile  Division 
Hartwell  Road 
Bedford,  Mass. 

Radio  Corporation  of  America 
Aviation  Systems  Laboratory 
225  Crescent  Street 
Waltham,  Maas. 


The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  California 
Attn:  Dr.  W.  C.  Hoffman 
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Wheeler  Laboratories,  Inc, 

122  Cutter  Mill  Road 
Great  Neck,  New  York 
Attn:  Mr,  Harold  A,  Wheeler 

Zenith  Plastics  Co. 

Box  91 

Gardena,  California 
Attn:  Mr.  S.  S.  Oleesky 

Library  Geophysical  Institute 
of  the  University  of  Alaska 
College 
Alaska 

University  of  California 
Berkeley  U,  California 
Attn:  Dr.  Samuel  Silver, 

Prof.  Engineering  Science 
Division  of  Elec.  Eng.  Electronics 
Research  Lab. 

University  of  California 
Electronics  Research  Lab «. 

332  Cory  Hall 
Berkeley  U,  California 
Attn:  J*  R.  Wninriery 

California  Institute  of  Technology 
Jet  Propulsion  Laboratory 
b800  Oak  Grove  Drive 
Pasadena,  California 
Attn:  Mr,  I.  E.  Newlan 

California  Institute  of  Technology 
1201  E.  Calif  omi  a  Street 
Pasadena,  California 
Attn:  Dr.  C.  Papas 

Carnegie  Institute  of  Technology, 

Schenley  Park 

Pittsburgh  13,  Pennsylvania 
Attn:  Prof.  A.  E.  Heins 

Cornell  University 
School  of  Electrical  Engineering 
Ithaca,  New  York 
Attn:  Prof.  G.  C.  Dalman 

University  of  Florida 
Department  of  Electrical  Engineering 
Gainesville,  Florida 
Attn:  Prof.  M.  H.  Latour,  Library 

Library 

Georgia  Institute  of  Technology 
Engineering  Experiment  Station 
Atlanta,  Georgia 

Attn:  Mrs.  J.H.  Crocland,  Librarian 

Harvard  University 
Technical  Reports  Collection 
Gordon  McKay  Library,  303 A  Pierce  ilall 
Oxford  Street,  Cambridge  38,  Mass. 

Attn:  Mrs.  E.L.  Huf schmidt,  Librarian 

Harvard  College  Observatory 
60  Garden  Street 
Cambridge  39,  Mass. 

Attn:  Dr.  Fred  L.  Whipple 

University  of  Illinois 
Documents  Division  Library 
Urbana;  Illinois 

University  of  Illinois 
College  of  Engineering  • 

Urbana,  Illinois 

Attn:  Dr.  P.  E.  Moyes,  Department  of 
Electrical  Engineering 

The  Johns  Hopkins  University 
Homewood  Campus 
Department  of  Physics 
Baltimore  18,  Maryland 
Attiu  Dr.  Donald  E.  Kerr 

Sandia  Corporation 
Attn:  Organization  l!i23 
Sandia  Base 

Albuquerque,  New  Mexico 


Applied  Physics  Laboratory 
The  Joh® Hopkins  University 
8621  Georgia  Avonue 
Silver  Spring,  Maryland 
Attn:  Mr.  George  L.  Seielstad 

Massachusetts  Institute  of  Technology 
Research  Laboratory  of  Electronics 
Room  2 OB-221 

Cambridge  39,  Massachusetts 
Attn:  John  H.  Hewitt 

Massachusetts  Institute  of  Technology 
Lincoln  Laboratory 
P.  0.  Box  73 
Lexington  73,  Mass. 

Attn:  Document  Room  A-229 

University  of  Michigan 
Electronic  Defense  Group 
Engineering  Research  Institute 
Ann  Arbor,  Michigan 
Attn:  J.  A,  Boyd,  Supervisor 

University  of  Michigan 
Engineering  Research  Institute 
Radiation  Laboratory 
Attn:  Prof.  K.  M.  Siegel 
912  N.  Main  St., 

Ann  Arbor,  Michigan 

University  of  Michigan 
Engineering  Research  Institute 
Willow  Run  Laboratories 
Willow  Run  Airport 
Ypsilanti,  Michigan 
Attn:  Librarian 

University  of  Minnesota 

Minneapolis  lU,  Minnesota 

Attn:  Mr.  Robert  H.  Stumm,  Library 

Northwestern  University 
Microwave  Laboratories 
Evanston,  Illinois 
Attn:  R.  E.  Beam 

Ohio  State  University  Research  Found. 
Ohio  State  University 
Columbus  io,  Ohio 
Attn:  Dr.  T.S.  Tice 

Dept,  of  Elec.  Engineering 

The  University  of  Oklahoma 
Research  Institute 
Norman,  Oklahoma 

Attn:  Prof.  C.  L.  Farrar,  Chairman 
Electrical  Engineering 

Polytechnic  Institute  of  Brooklyn 
Microwave  Research  Institute 
55  Johnson  Street 
Brooklyn,  New  York 
Attn:  Dr.  Arthur  A.  Oiiner 

Polytechnic  Institute  of  Brooklyn 
Microwave  Research  Institute 
55  Johnson  Street 
Brooklyn,  New  York 
Attn:  Mr.  A.  E.  Laemmel 

Syracuse  University  Research  Institute 
Collendale  Campus 
Syracuse  10,  New  York 
Attn:  Dr.  C.  S.  Grove,  Jr. 

Director  of  Engineering  Research 

The  University  of  Texas 
Elec.  Engineering  Research  Laboratory 
P.  0.  Box  8026,  University  Station 
Austin  12,  Texas 
Attn:  Mr.  John  R.  Gerhardt 
Assistant  Director 

The  University  of  Texas 
Defense  Research  Laboratory 
Austin,  Texas 

Attn:  Claude  W,  Horton,  Physics  Library 

University  of  Toronto 

Department  of  Electrical  Engineering 

Toronto,  Canada 

Attn:  Prof.  G.  Sinclair 


Lowell  Technological  Institute 

Research  Foundation 

P.  0.  Box  709,  Lowell,  Mass. 

Attn:  Dr.  Charles  R.  Mingins 

University  of  Washington 
Department  of  Electrical  Engineering 
Seattle  5,  Washington 
Attn:  G.  Held,  Associate  Professor 

Stanford  University 
Stanford,  California 
Attn:  Dr.  Chodorow 

Microwave  Laboratory 

Physical  Science  Laboratory 
New  Mexico  College  of  Agriculture 
and  Mechanic  Arts 
State  College,  New  Mexico 
Attn:  Mr.  H.  W.  Haas 

Brown  University  i 

Department  of  Electrical  Engineering 
Providence,  Rhode  Island 
Attn:  Dr.  C.  M.  Angulo 

Case  Institute  of  Technology 

Cleveland,  Ohio 

Attn:  Prof.  S.  Seeley 

Columbia  University 

Department  of-  Electrical  Engineering 

Morning 8 ide  Heights 

New  York,  N.  Y. 

Attn:  Dr.  Schlesinger 

McGill  University 

Montreal,  Canada 

Attn:  Prof.  G.  A;  Woonton 

Director,  The  Eaton  Electronics 
Research  Lab. 

Purdue  University 

Department  of  Electrical  Engineering 
Lafayette,  Indiana  •  .... 

Attn:  Dr.  Schultz 

The  Pennsylvania  State  University 
Department  of  Electrical  Engineering 
University  Park,  Pennsylvania 

University  of  Pennsvlvania 
Institute  of  Cooperative  Research 
3U00  Walnut  Street 
Philadelphia,  Pennsylvania 
Attn:  Dept,  of  Electrical  Engineering 
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